Lillehammer
PersonalSum: A User-Subjective Guided Personalized Summarization Dataset for Large Language Models
Zhang, Lemei, Liu, Peng, Henriksboe, Marcus Tiedemann Oekland, Lauvrak, Even W., Gulla, Jon Atle, Ramampiaro, Heri
With the rapid advancement of Natural Language Processing in recent years, numerous studies have shown that generic summaries generated by Large Language Models (LLMs) can sometimes surpass those annotated by experts, such as journalists, according to human evaluations. However, there is limited research on whether these generic summaries meet the individual needs of ordinary people. The biggest obstacle is the lack of human-annotated datasets from the general public. Existing work on personalized summarization often relies on pseudo datasets created from generic summarization datasets or controllable tasks that focus on specific named entities or other aspects, such as the length and specificity of generated summaries, collected from hypothetical tasks without the annotators' initiative. To bridge this gap, we propose a high-quality, personalized, manually annotated abstractive summarization dataset called PersonalSum. This dataset is the first to investigate whether the focus of public readers differs from the generic summaries generated by LLMs. It includes user profiles, personalized summaries accompanied by source sentences from given articles, and machine-generated generic summaries along with their sources. We investigate several personal signals -- entities/topics, plot, and structure of articles--that may affect the generation of personalized summaries using LLMs in a few-shot in-context learning scenario. Our preliminary results and analysis indicate that entities/topics are merely one of the key factors that impact the diverse preferences of users, and personalized summarization remains a significant challenge for existing LLMs.
Truncated Kernel Stochastic Gradient Descent on Spheres
Inspired by the structure of spherical harmonics, we propose the truncated kernel stochastic gradient descent (T-kernel SGD) algorithm with a least-square loss function for spherical data fitting. T-kernel SGD employs a "truncation" operation, enabling the application of series-based kernels function in stochastic gradient descent, thereby avoiding the difficulties of finding suitable closed-form kernel functions in high-dimensional spaces. In contrast to traditional kernel SGD, T-kernel SGD is more effective in balancing bias and variance by dynamically adjusting the hypothesis space during iterations. The most significant advantage of the proposed algorithm is that it can achieve theoretically optimal convergence rates using a constant step size (independent of the sample size) while overcoming the inherent saturation problem of kernel SGD. Additionally, we leverage the structure of spherical polynomials to derive an equivalent T-kernel SGD, significantly reducing storage and computational costs compared to kernel SGD. Typically, T-kernel SGD requires only $\mathcal{O}(n^{1+\frac{d}{d-1}\epsilon})$ computational complexity and $\mathcal{O}(n^{\frac{d}{d-1}\epsilon})$ storage to achieve optimal rates for the d-dimensional sphere, where $0<\epsilon<\frac{1}{2}$ can be arbitrarily small if the optimal fitting or the underlying space possesses sufficient regularity. This regularity is determined by the smoothness parameter of the objective function and the decaying rate of the eigenvalues of the integral operator associated with the kernel function, both of which reflect the difficulty of the estimation problem. Our main results quantitatively characterize how this prior information influences the convergence of T-kernel SGD. The numerical experiments further validate the theoretical findings presented in this paper.
Text mining in education
Ferreira-Mello, R., Andre, M., Pinheiro, A., Costa, E., Romero, C.
The explosive growth of online education environments is generating a massive volume of data, specially in text format from forums, chats, social networks, assessments, essays, among others. It produces exciting challenges on how to mine text data in order to find useful knowledge for educational stakeholders. Despite the increasing number of educational applications of text mining published recently, we have not found any paper surveying them. In this line, this work presents a systematic overview of the current status of the Educational Text Mining field. Our final goal is to answer three main research questions: Which are the text mining techniques most used in educational environments? Which are the most used educational resources? And which are the main applications or educational goals? Finally, we outline the conclusions and the more interesting future trends.
A Taxonomy of Information Attributes for Test Case Prioritisation: Applicability, Machine Learning
Ramรญrez, Aurora, Feldt, Robert, Romero, Josรฉ Raรบl
Most software companies have extensive test suites and re-run parts of them continuously to ensure recent changes have no adverse effects. Since test suites are costly to execute, industry needs methods for test case prioritisation (TCP). Recently, TCP methods use machine learning (ML) to exploit the information known about the system under test (SUT) and its test cases. However, the value added by ML-based TCP methods should be critically assessed with respect to the cost of collecting the information. This paper analyses two decades of TCP research, and presents a taxonomy of 91 information attributes that have been used. The attributes are classified with respect to their information sources and the characteristics of their extraction process. Based on this taxonomy, TCP methods validated with industrial data and those applying ML are analysed in terms of information availability, attribute combination and definition of data features suitable for ML. Relying on a high number of information attributes, assuming easy access to SUT code and simplified testing environments are identified as factors that might hamper industrial applicability of ML-based TCP. The TePIA taxonomy provides a reference framework to unify terminology and evaluate alternatives considering the cost-benefit of the information attributes.
CatchBackdoor: Backdoor Testing by Critical Trojan Neural Path Identification via Differential Fuzzing
Jin, Haibo, Chen, Ruoxi, Chen, Jinyin, Cheng, Yao, Fu, Chong, Wang, Ting, Yu, Yue, Ming, Zhaoyan
Abstract--The success of deep neural networks (DNNs) in real-world applications has benefited from abundant pre-trained models. However, the backdoored pre-trained models can pose a significant trojan threat to the deployment of downstream DNNs. Existing DNN testing methods are mainly designed to find incorrect corner case behaviors in adversarial settings but fail to discover the backdoors crafted by strong trojan attacks. Observing the trojan network behaviors shows that they are not just reflected by a single compromised neuron as proposed by previous work but attributed to the critical neural paths in the activation intensity and frequency of multiple neurons. This work formulates the DNN backdoor testing and proposes the CatchBackdoor framework. Via differential fuzzing of critical neurons from a small number of benign examples, we identify the trojan paths and particularly the critical ones, and generate backdoor testing examples by simulating the critical neurons in the identified paths. Extensive experiments demonstrate the superiority of CatchBackdoor, with higher detection performance than existing methods. CatchBackdoor works better on detecting backdoors( 1.5) by stealthy blending and adaptive attacks, which existing methods fail to detect. Moreover, our experiments show that CatchBackdoor may reveal the potential backdoors of models in Model Zoo.
Bletchley Park code machine that Hitler and generals used found rusting in Essex shed
Historians discovered a code machine used by Adolf Hitler to swap top secret messages with his generals when they saw it advertised on eBay for 9.50. Volunteers from the National Museum of Computing at Bletchley Park tracked down the extremely rare Lorenz keyboard after seeing it on the online bidding site. It was being advertised as a telegram machine and the historians found that it had been left in a shed in Southend-on-Sea, Essex, with'rubbish all over it'. Historians discovered a code machine used by Adolf Hitler to swap top secret messages with his generals when they noticed it was being sold on eBay. John Wetter, a volunteer at the museum, said: 'My colleague was scanning eBay and he saw a photograph of what seemed to be the teleprinter.
Applied AI News
The Hong Kong-based Mass Transit Railway Corp. (MTRC) has developed the Station Management Expert e Norwegian Police Data Center help predict aircraft fires and other System (SMES). SMES is an intelligent utilized an expert system to catastrophes. The police put and risk factors from the records functions and advising the controller the intelligent application online to of the National Transportation Safety of actions to take in case of emergency. The system is installed in Ya Ma at the games while complying with Carnegie Group and Westinghouse Tei Station as a test site, and the complex national employment regulations. Electric (both in Pittsburgh, Penn.) are MTRC plans to expand its use Plans are to deploy and network working with Pittsburgh area medical throughout the subway system as it the expert system into every law centers to develop an intelligent proves to be successful. The network Martin Marietta (Bethesda, Md.) is developed a neural network application will gather and organize data on using a real-time expert system to that has improved the efficiency clinical diagnoses, treatment, clinical build the Traffic Operations Center of its direct mail marketing efforts by and research findings, and patient (TOC) component of its Intelligent 35%.